milium 



United States Patent [19] 

Morris et al. 



US005862372A 
[li] Patent Number: 
[45] Date of Patent: 



5,862,372 
Jan. 19, 1999 



[54] VISUALLY ORIENTED COMPUTER 
IMPLEMENTED APPLICATION 
DEVELOPMENT SYSTEM UTILIZING 
STANDARDIZED OBJECTS AND MULTIPLE 
VIEWS 

[76] Inventors: Robert M. Morris, 201 Three Rivers 
Ct., Berwyn, Pa. 19312; Leet E. 
Denton, III, 148 Stoneway La., Bala 
Cynwood, Pa. 19004 

[21] Appl. No.: 340,702 

[22] Filed: Nov. 16, 1994 

[51] Int. CI. 6 G06F 9/06; G06F 9/22 

[52] U.S. CI 395/701; 345/335 

[58] Field of Search 395/650, 700, 

395/725, 701-710, 683; 364/200; 345/333, 

967, 335 

[56] References Cited 

U.S. PATENT DOCUMENTS 



4,455,619 6/1984 Masui et al 364/900 

5,019,961 5/1991 Addesso et al 364/192 

5,187,788 2/1993 Marmelstein 395/700 

5,191,645 3/1993 Carlucci et al 395/159 

5,325,481 6/1994 Hunt 395/159 

5,404,131 4/1995 Torres 364/521 

5,410,648 4/1995 Pazel 395/158 

5,442/744 8/1995 Piech et al 395/154 

5,446,548 8/1995 Gerig 356/375 

5,455,948 10/1995 Poole 395/650 

5,455,951 10/1995 Bolton ct al 395/700 

5,470,218 11/1995 Hillman 425/144 

5,481,712 1/1996 Silver 395/700 

5,519,828 5/1996 Rayner 395/161 

5,519,867 5/1996 Moeller et al 395/700 

5,544,301 8/1996 Orton et al 395/157 

5,555,369 9/1996 Menendez et al 395/161 



5,675,752 10/1997 Scott et al 395/333 

OTHER PUBLICATIONS 

Power Biulder, User's Guide Version 3.0, Powersoft Enter- 
prise Series, Chapter 10, pp. 255-287, Aug. 1993. 
Gibbs, S., Composites Multimedia and Active Objects, 
OOPSLA '91, 1991. 

Emery, J. et al, "A Multimedia Document Model for Con- 
tinuous Media", IEEE Ondisc, pp. 640-643, 1993. 
V__Graph Inc, Ozone manual, Version 1.5, 1995. 
E. Brown, "Authoring Programs", New Media Tool Guide 
1995, pp. 17-25, 1995. 

Primary Examiner — Alvin E. Oberley 

Assistant Examiner — Sue Lao 

Attorney, Agent, or Firm — Laurence A. Weinberger 

[57] ABSTRACT 

A computer implemented application development 
(authoring) system permits objects (such as VBX custom 
controls) to be graphically inserted into the program under 
development by dragging and dropping associated icons into 
one of four views. The properties associated with the object 
may then be assigned settings. Development of a complete 
application is accomplished by visually arranging, ordering, 
and interconnecting the objects without the necessity of 
writing any code. The four views of Output, Map, 
Multitrack, and Workform may be synchronized so that 
changes made to the program in one view are simulta- 
neously reflected in all other views. The system generates as 
output a script listing the objects and their properties which 
is then executed by a separate run time program. The system 
permits use of objects written to a standard specification and 
the addition at any time of additional objects written to that. 
Integration of the objects into the system is achieved by 
wrapping each object in an "envelope" of system specific 
properties. 

4 Claims, 6 Drawing Sheets 
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VISUALLY ORIENTED COMPUTER (parameters) they required for their use. Thus, applications 

IMPLEMENTED APPLICATION would be built by selecting appropriate software modules 

DEVELOPMENT SYSTEM UTILIZING assembling/ordering/sequencing them to achieve higher 

STANDARDIZED OBJECTS AND MULTIPLE order factions. This is the essential goal of "object oriented 

VIEWS S programming". It quickly became apparent that building 

with modularized software would not be efficient unless the 

FIELD OF THE INVENTION presentation of the modules themselves conformed to some 

form of standard; i.e., flat bricks of any material can be 

Hiis invention relates to the use of a computer imple- stacked, mixed flat and curved bricks, no matter what 

mented application development process, and, more material they are made of, do not stack well. A second 

specifically, to an authoring system and its associated run problem facing object oriented programming is that effi- 

time system which permits the creation of an application ciencies of application construction would only be achieved 

(program) in which standardized objects are incorporated when thc assembly of the objects did not require extensive 

into the application without the necessity of the user having ^ com plex programming itself 

knowledge any programming language or programming Several attempts have been m ^ ^niDie, to devel 

script^ndinwhichtheapphcationdevelopmentprocesscan yisuall oriented prograrnmillg systems in which so ft ware 

be monitored (observed, evaluated, etc.) through the mul- modules are manipulated as icons which be arranged/ 

tiple synchronized views of output display, timing sequence, ordered as needed and i Qtcrconn ected with logical operators, 

logical relationships, and script workform. Examples of this can be found in the NeXT application 

BACKGROUND OF THE INVENTION 20 develo P ment system, in the VIP Macintosh program, and in 

PowerBuilder, a Microsoft Windows program. Generally, in 
The last ten years has seen an explosive growth in the these types of application development systems, the soft- 
development of digital computer systems and technology ware modules or objects are presented on a palette from 
both in the processing power and of the hardware systems which they are selected and "dragged" into the application 
and in the sophistication of the controlling software. Not 25 under development. Ideally, in these systems, the software 
only has the microprocessor become a ubiquitous incident to instructions needed to integrate the objects into a function- 
almost every laboratory, industrial, and household ing whole (corresponding to the visual representation or 
appliance, but many functions once assigned to combina- map) are provided by the authoring program system. In 
tions of moving parts are almost invariably now preformed reality, most of the prior art systems require the user to write 
by digitally controlled electronics. Systems as diverse as 30 some of the interconnecting software code either in a 
automobile ignition systems and home turntables now share common higher order language or in a language specialized 
a common electronic heritage. In fact, the performance of to that program. 

many electronic circuits can be equally implemented by \ t was hoped that the demand for complex and varied 
hardware or by a digital computer under software control. In software imposed by multimedia programming require- 
many instances the software component of the digital elec- 35 men ts could be efficiently addressed by the use of object 
tronics has become enormously complex and correspond- oriented programming. Until the present invention, these 
ingly expensive to develop. Indeed, the cost of the electronic hopes have not been realized. Amajor problem not solved by 
hardware has come down so far that the development time the prior art has been the inability to handle in one visual, 
and cost of the software is now the limiting element in the non-scripted, application development system the diverse 
introduction and pricing of many new systems. There has 40 requirements of different types of data required by a multi- 
developed, therefore, a great need to reduce the cost and med j a pr0 gram in appropriate visual interfaces, 
time of software development. For during the authoring process a view of the 

The development over the past three years of what is output generated by the application under development is 

referred to as "multimedia" computer systems has increased useful when dealing with visual output (images). However, 

the complexity of the required software as the systems 45 such a view is much less useful for dealing with audio output 

access, simultaneously present, and then replace vast since, clearly, audio can not be seen. Correspondingly, a time 

amounts of information. Present systems are directed toward li ne view in which the elements of the application program 

the simultaneous presentation of audio and visual output, but are represented by graphs plotting the relative times and 

systems are clearly on the horizon which will present sequences of presentation is much more useful for audio, 

coordinated output which can be sensed by all five human 50 Similarly, a logical map (flowchart paradigm), which visu- 

senses and create a virtual reality. While present systems aU y relates the sequence and logical interconnections of 

access the information for multimedia presentation prima- each program element (visual, audio, etc.) is very useful for 

rily from local magnetic and optical storage media, it is designing and following the basic structure of the applica- 

foreseeable that electronic and optical communication with tion program. Sometimes, for quickly manipulating or edit- 

the world through the coming "information superhighway" 55 ing particular parts of the developed application code, it is 

will provide a rich source of interactive multimedia more useful to be able to view the program in a text view and 

In an attempt to reduce the cost and complexity of treat the underlying script like a text document. Not only 

developing sophisticated software, attempts have been made should the application development system provide, at any 

over the last few years to modularize software functions given moment, a view appropriate to the type of multimedia 

with the idea that such modularized software could then be 60 material being utilized, but such views should be able to be 

integrated into larger systems to achieve more complex synchronized so that changes made in one view are instantly 

functions. Each piece (object) of modularized software reflected in all the others. Prior art application development 

would consist of a set of instructions designed to perform a programs do not have the flexibility to present all four views, 

specific type of task which was frequendy required in a more much less in a synchronized manner. Programs are available 

complex program. To use the objects, no knowledge of their 65 which are optimized for output view, map view, timeline 

internal coding structure would be necessary. All that would view, and script view. Some permit more than one view with 

be required is to provide the modules with the information some synchronization. However, no prior art program per- 
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raits access to all four synchronized views. In particular, no FIG. 3 shows a Multitrack view of a simple program. 

prior art program has successfully overcome the difficulty of p IG 4 shows a Workform view of a simple program. 

simultaneously synchronizing the time lice view to the other - , f , u f . „ n llt ^ llt 

3 3 6 FIG. 5 shows for a simple program all four views, Output, 

Views " Map, Multitrack, and Workform simultaneously displayed in 

Additionally, in an optimum visual, object oriented, appli- 5 separate windows. 

cation development program there should be available a CTO , « , • 1 1 A - i • *u 

., • * r i_- * / & j i . u P IG - 6 shows a typical palette displaying the objects 

wide variety ot objects (software modules that would u . t U1 /\ u * iU ' 

■ , , . \ 1t . j. , v , which are available to the user of the system, 
manipulate m different manners multimedia output) with the 

ability to easily add additional objects, developed by inde- DETAILED DESCRIPTION OF THE 

pendent programmers, to the development system. Each 10 INVENTION 
added object should seemlessly be integrated so that it is 

available in all four views. Programs of the prior art each Definitions 

have their own object definition standards. That is, for an r , i . , it . . „ , . t . 

,. . , . ■ n . , i U - 4 . **u * In order to avoid the confusion often present in the world 

object to be incorporated into their system, it must meet that f 4 e c £• v r *u 

program's standards. No prior art program is available 15 of computer jargon for purposes of this application, the 

which will allow addition of objects not designed to its mowm * terms sha11 have the meamn S S set forth below: 

standard without the necessity of writing additional code to "APPLICATION" shall mean an end product computer 

perform the interface to that object. Finally, it should be program which is ready for use by its intended audience, 

noted that application development systems necessarily con- "CODE" shall mean a series of specialized words and 

sist of two major parts. The first is the authoring section 20 symbols in a particular syntax which must be entered 

which provides a customized output (script). The second is through a keyboard and which is then converted to a 

a run time system which takes the customized output and machine readable form. 

executes it. Some prior art run time systems for visually "OBJECT" shall mean a modular software program writ- 
developed applications are notoriously slow in their execu- ten to a defined specification which may perform any 
tion. 25 function. For purposes of this application, the word 

"OBJECT* shall be considered synonymous with the term 

SUMMARY OF THE INVENTION "custom control". Objects are intended to be grouped and 

The present invention overcomes the limitations of the assembled into applications, 
prior art. First, the system can be used to author applications "SCRIPT" shall mean a set of computer instructions 
using an entirely visual programming scheme (paradigm) written at a higher level than code i.e., farther removed from 
which does not require the user to know or be able to write machine language, which is usually interpreted during 
any specialized code. Icons representing the objects (and execution. "SCRIPT" for purposes of this application shall 
accordingly their functionalities) may be placed (dragged) also refer to the listing of objects and property settings 
into an appropriate view. The system of the invention 35 generated by the authoring program of this invention, 
generates the information needed by the run time program. "SYSTEM" shall mean the computer implemented appli- 
Second, during authoring, the user can access all four views cation development system of this invention, 
which can be synchronized or not at the user's choice, and "WINDOWS" (when appearing in all capital letters) shall 
these views are available in overlapping windows. The user mean the Microsoft operating system of the same name 
may, thus, work in the view most appropriate to the type of 4Q without regard to the version number, 
object the user is manipulating. Tnird, the objects which are ^ present invention is an integrated computer imple- 
available on the palette, can be added or subtracted from the mC nted application development system which permits the 
development program with great ease. While the present user to create custom applications without writing a line of 
implementation of the invention is designed to directly computer code. The ability to do just this has existed as a 
operate standardized Microsoft Visual Basic VBX controls 45 dream computers moved into the public imagination 
as objects, the manner in which the invention wraps the and thc drcam is fulfilled by this invention. The invention 
objects is equally usable with objects written to standards provides a fully visual programming environment in which 
other than the Microsoft VBX control. Fourth, the invention me can create power ful WINDOWS applications in 
is capable of utilizing any objects written to a standard hours that would take days or weeks using standard pro- 
specification. That is, at its core it is featureless and exten- 5Q graiTITT1 in g languages. When describing the process imple- 
sible. The invention imposes no limitation on the type of me nted by a non-authoring computer system, it is generally 
function the objects can preform, and the objects need not be sufficient to describe the process steps carried out and their 
directed towards multimedia or any other single use. Thus, order However, authoring programs create an environment 
all types of programs can be developed utilizing the entire ^ many potentialities within which the user works to 
range of available objects, and, during application $$ develop the application. To describe authoring programs it is 
development, the objects may all be presented in the four necessary to describe the environment. Consider the analogy 
views, once again synchronized or not. Fourth, a run time lo buHding a house. The environment provides the tools 
program is provided which executes the output of the (hammers, nails, saws, lumber, wires, pipes, etc.) as well as 
authoring program much faster than the runtime programs of ways of looking at the house under construction 
the prior art. These, and other advantages of the invention 6Q (architectural plans, outside drawings, kitchen layouts, etc), 
will become apparent from the description of the invention 0nce the environment contains all the tools and views 
which follows. needed, the user can construct the house according to his/her 
BRIEF DESCRIPTION OF THE FIGURES own needs by assembling the parts. 

This description of the computer implemented application 

FIG. 1 shows a schematic representation of the major 55 development system will briefly: 1) describe the environ- 

components of a modern programmable digital computer. me nt created by the present invention; 2) describe the 

FIG. 2 shows a Map view of a simple program. process of using the system to create applications; 3) 
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describe the manner in which the system creates a script a view of just any part of the script (program), it only views 

corresponding to the program elements and how that script timed sections. This view is useful for creating parts of the 

is used by the run time program; and 4) describe how the application that require objects to be created and destroyed 

system coordinates objects and permits additional objects to at specific times. The Multitrack view automatically sets all 

be added easily. A complete description of the invention is S toe special properties that differentiate the objects in these 

contained in the source code provided in the software *! me sections form the objects in other parts of the applica- 

appendix which is attached to this application. ** on ( scr ipO* 

™„ . . 1 j- . . • c i FIG. 4 shows a Workform view of a simple program 

FIG. 1 is a block diagram showing the basic functional ^ ^ ^ ^ f d 

units of a modern digital computer with which the computer m each m ^ ^ Workform view displays the under i ying 

implemented applications development system of the inven- W ^ directly ^ u ^ & ^ docm{ 

tion works. Information may be stored on magnetic or The cm be yiewed in as Uuk or as much detail ^ is 

optical media for access or be available through a connec- necessar y. This view has word processing features so that it 

tion to a remote information source. During the development becomes easy to make global changes, and to copy and 

of an application with the system of this invention, the user move blocks of information. 

directs the incorporation of objects into one of the four ^ fig. 5 shows for a simple program all four views, Output, 
views, modifies the properties associated with that object, Map, Multitrack, and Workform simultaneously displayed in 
and interconnects the objects so chosen. The user may direct separate windows. In each view, the system of this invention 
the machine to accomplish these tasks using a pointer (such permits development of the application using the same 
as a mouse) or a keyboard and visually observes his/her method of adding an object, interconnecting the object, and 
actions on the system's output display. As the system 20 setting the objects properties. For instance, from a palette of 
responds to the user's directions, it creates a script repre- objects, which displays a collection of icons that represent 
sen ting the application being developed by the user. When each object available to the system (such as the palette 
the development process is complete, the user stores the shown in FIG. 6), the user of the system drags an object into 
script on one of the system's storage devices. the view. The system simultaneously generates the under- 
One of the interesting features of the object oriented 25 ^g. script which reflects the object and the properties 
A • • i i associated with the obiect. The user s double click on the 
authoring system of this invention is that no traditional icofl esentin the J object cnts a mcim list of the 

looking code, written in a programming language, is gen- t £ s for t f at objec J t th( f ^ of which can be 

erated by the system. Rather, the output of the authoring changed according to the ^ s requirements. If a separate 

system is a listing of the objects in order of appearance (m fa hg box is avai i a ble for a particular property, clicking on 

the program sequence) along with the properties associated the property gets the user to the dialog box. As a property 

with that object at that place in the sequence. The properties S6tting ^ defined? the system records tbat de finition in the 

are the information or settings which specify to the object associated underlying script. In addition, the system user can 

how it will perform. The computer implemented application elect t0 have ^ the four views synchronized so that each 

development system records the settings for the objects and view is updated f or every change made in any view to the 

the order of execution of the objects in an output script. underlying script. 

Thus, the system operates only at the object level. As the Fof cxampk in the Output view window the user might 

author proceeds in developing the application, the system dragan i conre p resen ting an object which pulls a picture into 

adds to the senpt. thc app h cat j ori program. The Output view window would 

The four views are different ways to make and look at the 4Q then display the picture. If several pictures were to be 

same script. Each has its own special perspective and presented in some order, several picture icons would be 

strength. FIG. 2 shows a Map view of a simple program in placed in the output view window. The user could set the 

which the flow of the program and the flow of data is order of presentation in the Workform view, but might more 

displayed. The Map view displays a block diagram in which graphically do so in the Map view. In the Map view the user 

the authored program is viewed as the sequence in which the 45 cou t d arrange the picture icons in the order in which he/she 

objects are utilized, the branches which are taken, and the wished the pictures to appear. In addition, in the Map view, 

flow of data between the objects. The system not only allows the user could add conditional branches from picture to 

the passing of data back and forth between different objects, picture. The display timing for each picture could be set 

but allows objects to affect each other in other ways as well. through the properties boxes or the Workform view, but 

The Map view is best for seeing and building the program 5Q might more graphically be done in the Multitrack view 

using a flowchart metaphor. Thinking of the program in this where each pictures duration and sequence would be indi- 

way can be very useful when building interactive applica- cated on parallel time lines. Finally, if the user wished to 

tions since this view presents a way to see the flow of data duplicate part of the application already created, he/she 

between objects which can save hours of debugging time. could use the Workform view to copy and move the under- 

When a Map window is opened, the system generates the 5S i y i ng scr j pt Clearly, the user of the system has the option of 

Map view directly from the underlying script. choosing the view most appropriate to the type of applica- 

Tbe Output view displays the screen output of the objects tion under development. Since the system is a general 

and is best for working directly with the product of the purpose programming environment, the user will choose the 

script. In this view you see the output of each object you view most appropriate for the application, 

incorporate into the program. When an Output view window 60 It should be noted that a very powerful feature of the Map 

is opened, the system generates the Output view directly view resides in its ability to permit the application developer 

from the underlying script. to specify that one object in the sequence get or set data 

FIG. 3 shows a Multitrack view of a simple program in from/to another object. Thus, information exchange among 

which the timing sequences of object presentation are dis- objects is enabled as well as logical branching and program 

played. The Multitrack view is limited to coordinating and 65 flow between objects. 

timing the appearance, disappearance, and activities of the In order to run a program developed in this manner from 

objects. It is different from the other views because it is not the script created by the authoring system, a run time 
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program is required which executes each object in the 
defined sequence with the defined property settings. The 
system of this invention provides such a run time program. 
Programs authored by the computer implemented applica- 
tion development system of this invention run faster under 
its run time program than to object oriented programs 
developed and run under Microsoft Visual Basic. 

The computer implemented application development sys- 
tem of this invention is able to integrate and work with the 
objects in the above described manner because, in addition 
to the internal properties which are unique to each object and 
relate to the special work that each object does, the system 
wraps all objects in a kind of "envelope" of special prop- 
erties that regulate how the objects act in the system. These 
special properties are: 



BranchTo 


Kill 


Break 


Label 


Comment 


TiiJieMark 


Create 


Track 


Create Type 


Type 


Destroy 


WaitFor 


Destroy Type 


Group 
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No other program in the prior art wraps standardized objects 
in this manner to achieve similar functionalities. This wrap- 
ping is also what permits the easy integration of additional 
objects into the system. Also, the system is so designed that 
the number of such special properties may be increased, if 
necessary, to adapt to any changes in future definition of 
objects. 

As noted earlier, there are many different standards for the 
format of objects which have been devised by the propo- 
nents of object oriented programming. Many prior art 
authoring systems use objects with their own proprietary 
standard. However the implementation of the authoring 
system of this invention disclosed in the software appendix 
has been designed to operate with "VBX custom controls" 
(objects) written to the Microsoft Visual Basic and Visual 
C++ standard. Presently, there are about 300 software ven- 
dors across the United States developing and selling such 
objects. Thus, a large number of objects written to the same 
standard are now available. The Microsoft Visual Control 
Pack contains representative objects and examples of code 
for different control types. Any object written to this stan- 
dard may be used with the system of this invention. 
However, the methods employed by the present invention to 
wrap objects and integrate and run them directly from the 
system are equally applicable to objects written to other 
standards, and the scope of the present invention should not 
be considered limited by reference to the VBX control type 
object. 

The source code in the software appendix does not 
contain any objects. As noted above, objects may be 
obtained through a large number of commercial suppliers. 
However, the system of this invention is sufficiently general 
to permit development of applications of all types. Special- 
ized applications may be developed by importing appropri- 
ate standardized objects into the system. Thus, the system is 
infinitely extensible since it provides the necessary back- 
bone environment for application development with objects 
written to a common standard. 

While the software appendix discloses the source code for 
the computer implemented application development system 
of this invention in the WINDOWS and IBM-PC platform, 
the invention is not platform dependant and could also be 
implemented on the Macintosh and other platforms. In order 



to run the source code disclosure of this invention on a 
digital computer, the following commercially available soft- 
ware programs should be available on the computer: 
Microsoft Visual C++ 1.5, Microsoft WINDOWS 3.1, and 
objects of the user's choice written to the Microsoft VBX 
Custom Control standard, Version 1.0. For Microsoft Visual 
C++ 1 .5, file VBAPI.H has been renamed to VB_VB API.H. 

The computer implemented application development sys- 
tem of this invention is presently available from 
V^GRAPH, Inc. under the trademark OZONE. 

What is claimed is: 

1. A computer implemented application development sys- 
tem for visually developing an application without the 
necessity of writing any code comprising: 

a. means for wrapping objects written to a standard 
specification; 

b. means for establishing four views; 

c. means to synchronize said views; 

d. means for moving an object into one of four said views; 

e. means for setting the properties of said object; 

f. means for moving and setting the properties of addi- 
tional objects until all of the desired objects have been 
specified; 

means for interconnecting the objects in temporal 
sequences; 

h. means for specifying the flow of data and control 
between the objects; 

i. means for generating a script reflecting means C. 
through H.; and 

j. means for running said script. 

2. The system of claim 1 in which the objects written to 
a standard specification are VBX Custom Controls. 

3. A computer implemented application development sys- 
tem comprising: 

a. means for storing objects written to a standard speci- 
fication; 

b. wrapper means for encapsulating said stored objects; 

c. means for visually displaying the application under 
development comprising: 

(1) means for displaying an output view; 

(2) means for displaying a map view; 

(3) means for displaying a multitrack view; and 

(4) means for displaying a workform view; 

d. means for providing synchronization among four said 
visual display means; 

e. means for selecting from said object storage means one 
or more of said wrapped objects for inclusion into the 
application under development; 

f. means for setting the properties of each selected 
wrapped object; 

g. means for setting the desired temporal relationships 
among said wrapped objects; 

h. means for specifying the flow of data and control 
among said selected wrapped objects; 

i. means for creating and storing a persistent representa- 
tion of said selected wrapped objects, the property 
settings of said objects, the temporal relationships 
among said objects, and the flow of data and control 
among said objects; and 

j. means for executing said representation. 

4. The system of claim 3 in which the objects written to 
a standard specification are VBX Custom Controls. 
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